The present invention relates generally to image processing, and, more particularly, to video surveillance applications and methods of extracting a foreground object from a background image.
Image processing methods have various applications, many of which may be applied to applications such as video surveillance and other security-related purposes. Taking video surveillance as an example, closed-loop video monitoring systems have been used for security-related purposes over the past few decades. However, these systems may be limited to recording images in places of interest, and do not support analysis of objects or events. With the development and advancement in digital video and automatic intelligence techniques, intelligent monitoring systems based on computer vision have become popular in the security field. For example, intelligent surveillance systems may be deployed in airports, metro stations, banks or hotels for identifying terrorists or crime suspects. An intelligent monitoring system may refer to one capable of automatically analyzing images taken by cameras without manual operation for identifying and tracking mobile objects such as people, vehicles, animals or articles. In analyzing the images, it may be helpful or necessary to distinguish a foreground object from a background image to enable or simplify subsequent analysis of the foreground object.
Conventional techniques for extracting foreground objects may include background subtraction, temporal differencing and optical flow. The background subtraction approach may include a learning phase and a testing phase. During the learning phase, a plurality of pictures free of foreground objects may be collected and used as a basis to establish a background model. Pixels of the background model may generally be described in a simple Gaussian Model or Gaussian Mixture Model. In general, a smaller Gaussian model value may be assigned to a pixel that exhibits a greater difference in color or grayscale level from the background image, while a greater Gaussian model value may be assigned to a pixel that exhibits a smaller difference in color or grayscale level from the background image. An example of the background subtraction approach can be found in “A System for Video Surveillance and Monitoring” by Collins et al, Tech. Rep., The Robotics Institute, Carnegie Mellon University, 2000. In certain applications, the background subtraction approach may be disadvantageous in extracting foreground objects that may have a color closer to that of background. Furthermore, a shadow image may be incorrectly identified as a foreground object in some applications, and the change in hue may adversely affect the extraction sometimes. Consequently, the resultant picture extraction may be relatively broken or even unrecognizable.
As to the temporal differencing approach, it may directly subtract pictures taken at different timings. A pixel may be identified as a foreground pixel of a foreground object if the absolute value of a difference at the pixel point between the pictures exceeds a threshold. Otherwise, the pixel may be identified as a background pixel. An example of the temporal differencing approach may be found in “Change Detection and Tracking Using Pyramid Transformation Techniques” by Anderson et al, In Proc. of SPIE Intelligent Robics and Computer Vision, Vol. 579, pp. 72-78, 1985. Depending on its applications, the temporal differencing approach may be disadvantageous in extracting foreground objects that may be immobilized or move relatively slowly across the background. In general, local areas having boundaries or lines of a foreground object may be easily extracted. Block images of a foreground object without significant change in color, for example, the close-up of clothing, pants or faces, however, may be incorrectly identified as background images in some applications.
The optical flow approach, based on the theory that optical flow changes when a foreground object moves into background, may calculate the amount of displacement between frames for each pixel of an image of a moving object, and determine the position of the moving object. An example of the optical flow approach may be found in U.S. Published patent Application No. 20040156530 by Brodsky et al., entitled “Linking tracked Objects that Undergo Temporary Occlusion.” In some examples, the optical flow approach may involve a relatively high amount of computation and may not support real-time image processing.
Therefore, it may be desirable to have image processing methods that may alleviate some or all of the disadvantages of the conventional approaches. In some applications, it may also be desirable to have methods that may extract a foreground object in multiple stages and update a background model in order to enhance the extraction ability.